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EFFECTS  OF  KNOWLEDGE  TYPES  ON  OBJECTIVE 
AND  PERCEIVED  CHOICE  PERFORMANCE 

Abstract 

Two  components  of  knowledge  used  in  making  choices  are  examined:  alternative- 
specific  knowledge  and  choice  rule  knowledge.  The  effects  of  these  knowledge  types, 
both  singly  and  jointly,  upon  objective  choice  quality  and  perceived  choice  quality  are 
examined  in  two  studies.  The  results  of  Study  One  indicate  that  the  knowledge  types  are 
differentially  beneficial,  and  that  subjects  tend  to  be  more  overconfident  about  the  perceived 
quality  of  their  choices  when  they  have  alternative-specific  knowledge  than  when  they  have 
choice  rule  knowledge.  The  hypothesis  that  this  difference  is  due  to  subjects'  greater 
awareness  of  alternative-specific  knowledge,  rather  than  rule  knowledge,  is  examined  and 
supported  in  Study  Two. 


EFFECTS  OF  KNOWLEDGE  TYPES  ON  OBJECTIVE 
AND  PERCEFV^ED  CHOICE  PERFORMANCE 

Eloise  Coupey  and  Sunder  Narayanan 
University  of  Illinois  at  Urbana-Champaign 

Knowledge  is  a  frequent  and  desirable  component  of  decision  making  for  choice. 
Knowledge  can  be  used  to  guide  the  acquisition  and  organization  of  relevant  information 
and  aid  in  the  selection  of  an  appropriate  rule  for  evaluating  options  for  choice.   However, 
despite  much  research  on  decision  making  for  choice,  questions  still  remain.  For  example, 
can  the  knowledge  that  influences  decision  making  for  choice  be  classified  into  different 
types?  If  so,  are  some  types  more  beneficial  than  others?  Are  decision  makers  aware  of 
different  types  of  knowledge?  Do  they  recognize  that  one  type  may  be  more  useful  for 
making  a  choice  of  good  quality  than  another  type? 

The  research  described  in  this  paper  addresses  these  questions  about  knowledge, 
building  upon  past  research  in  decision  making  and  problem  solving.  In  subsequent 
sections  of  this  paper  we  present  a  theory-based  rationale  for  examining  two  types  of 
knowledge  that  are  relevant  for  making  a  choice:  alternative-specific  knowledge,  i.e., 
knowledge  about  the  alternatives,  and  rule  knowledge,  i.e.,  knowledge  of  a  choice  rule  for 
integrating  and  evaluating  information  about  alternatives.  Empirical  assessments  of  these 
two  knowledge  types  and  their  effects  on  objective  and  perceived  quality  of  choice  are 
obtained  with  two  studies.  The  results  of  these  studies  indicate  that  the  knowledge  types 
are  differentially  beneficial,  and  that  subjects  tend  to  be  more  overconfident  when  they  have 
alternative-specific  knowledge  than  when  they  have  rule  knowledge. 

BACKGROUND 

Anderson  (1983)  described  knowledge  as  consisting  of  declarative  knowledge  and 
procedural  knowledge.  Declarative  knowledge  consists  of  facts  about  the  items  in  a 
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problem  space,  such  as  the  attributes  of  the  alternatives  in  a  choice  problem,  and  the  values 
of  these  attributes  for  different  alternatives.  Procedural  knowledge  reflects  the  skill 
component:  knowing  how  to  use  the  declarative  infonnation.  The  rules  typically  ascribed 
to  decision  makers  for  arriving  at  choices  (e.g.,  elimination-by-aspects  (Tversky  1972))  are 
examples  of  procedural  knowledge.  They  direct  the  use  of  declarative  information. 

Newell  and  Simon  (1972)  describe  a  similar  dichotomy  with  their  distinction 
between  representation  and  method  for  problem  solving.  The  internal  representation 
reflects  the  current  situation  for  the  decision  maker.  It  is  contained  in  a  problem  space, 
along  with  all  other  possible  states  of  knowledge.  The  knowledge  states  are  nodes, 
connected  by  processes.  Processes  are  carried  out  by  the  activation  of  appropriate 
operators.  If  the  internal  representation  enables  the  decision  maker  to  select  and  apply  a 
method,  such  as  a  choice  rule  in  a  choice  problem,  which,  when  executed,  will  achieve  a 
desired  end,  the  decision  maker  carries  out  the  method.  If  no  method  is  appropriate  the 
decision  maker  changes  the  internal  representation,  adjusting  it  until  a  method  can  be 
retrieved  and  applied. 

A  third,  similar  distinction  is  made  by  Chi,  Glaser,  and  Rees  (1981)  in  their  study 
of  the  development  of  expertise.  They  describe  expertise  as  consisting  of  two  components: 
a  body  of  usable  information  and  procedural  skill.  They  suggest  that  what  distinguishes 
experts  from  novices  in  any  domain  is  that  experts  have  the  ability  to  convert  knowledge  of 
fundamental  principles  (as  in  physics  problems)  into  procedures  for  problem  solving.  That 
is,  a  large  amount  of  alternative-specific  information  win  not  automatically  make  a  decision 
maker  an  expert  decision  maker.  A  lot  of  information  may  be  useless  unless  the  decision 
maker  knows  how  to  evaluate  it 

The  knowledge  components  described  above  are  necessary  for  evaluating 
alternatives  to  make  a  choice.  It  must  be  noted,  however,  that  the  choice  process  exists 
beyond  the  stage  of  evaluating  alternatives.  For  example,  consider  the  process  model 
depicted  in  Figure  1.  The  stage  of  Need  Recognition  may  necessitate  knowledge  about 


needs  and  satisfactory  goals.  Information  Search  to  identify  alternatives  requires 
knowledge  about  existing  alternatives.  These  knowledge  types  may  differ  from  the  types 
of  knowledge  required  to  evaluate  alternatives  and  to  select  one  alternative.  We  focus  on 
the  evaluation  stage,  in  which  we  propose  that  two  types  of  knowledge  are  required  These 
types,  alternative-specific  knowledge  and  rule  knowledge,  reflect  the  dichotomy  proposed 
by  Anderson.  Both  types  of  knowledge  can  exist  independently  of  the  other.  Therefore, 
we  study  the  relative  impact  of  these  knowledge  types  on  objective  and  perceived  choice 
quality. 


Figure  1  about  here. 


Separation  of  Knowledge  Types 

Much  attention  has  been  focused  on  understanding  how  the  amount  and  structure  of 
information  in  long-term  memory  influences  choice  performance  (Bettman  1986).  For 
example,  consumer  researchers  have  looked  at  the  effects  of  product  knowledge  on 
information  search  (Bettman  and  Park  1980;  Brucks  1985;  Srull  1983),  and  on  evaluation 
processes  (Sujan  1985).  However,  alternative- specific  knowledge,  while  undoubtedly 
useful,  is  not  the  sole  contributor  to  superior  choices.  One  may  abstract  general  choice 
rules  which  can  be  transferred  from  one  domain  to  another.  To  understand  this,  consider  a 
choice  situation.  There  are  typically  two  tasks  associated  with  a  choice.  One  task  is  the 
extraction  of  relevant  information  about  alternatives  from  the  environment  The  other  task 
is  the  application  of  a  particular  choice  rule  to  this  relevant  information,  and  the  selection  of 
the  alternative  that  offers  the  highest  utility  to  the  decision  maker.  These  two  tasks  need  not 
be  sequential.  In  some  situations,  the  choice  rule  may  be  selected  first,  and  then  used  as  a 
template  for  guiding  information  acquisition  (Bettman  and  Kakkar  1977;  Lussier  and 
Olshavsky  1979).  In  othCT  situations,  information  may  be  acquired  prior  to  selection  of  an 
evaluative  rule  (Biehal  and  Chakravarti  1982).  Sometimes  the  two  tasks  may  even  occur 


simultaneously,  as  when  a  decision  maker  constructs  a  choice  rule  at  the  time  of 
infonnation  evaluation  (Bettman  and  Zins  1977;  Hayes-Roth  and  Hayes-Roth  1979). 

That  choice  rules  can  be  developed  and  used  independently  of  alternative-specific 
information  is  supported  by  research  on  problem-solving  abilities  in  psychology.  Anzai 
and  Simon  (1979)  develop  a  theory  of  'learning  by  doing',  in  which  they  argue  that  people 
learn  which  rules  are  most  appropriate  for  problem  solving  in  a  particular  domain  by  a 
process  of  trial  and  error.  Anzai  and  Simon  draw  a  clear  distinction  between  knowledge  of 
alternative-specific  information  (such  as  familiarity  with  the  problem  representation)  and  the 
processes  used  to  solve  the  problem  (described  as  the  general  learning  capabilities  of  their 
model)  ^.  This  means  that  one  can  learn  a  rule  in  one  domain,  and  then  retain  knowledge  of 
that  rule  for  use  in  a  different  domain.  Anzai  and  Simon  refer  to  this  type  of  strategy  as  one 
that  is  'task  independent'.  This  depiction  is  consistent  with  earlier  work  by  Newell  and 
Simon  (1972),  who  describe  two  main  elements  of  problems  and  their  solution:  the 
representation  of  information  and  the  method  used  to  process  it  The  method  can  be 
specified  independently  of  task  information  and  can  be  used  to  process  a  variety  of  task 
infonnation. 

The  implications  of  task  separation  in  knowledge  development  are  important  For 
example,  in  many  situations  people  are  passively  exposed  to  information  without  a  goal  for 
encoding  information  (e.g.,  exposure  to  advertisements).  Some  of  the  information  may  be 
encoded  in  long-term  memory  (Gordon  and  Holyoak  1983).  Consequently,  over  a  period 
of  time  a  decision  maker  can  become  knowledgeable  about  the  alternatives  in  a  particular 
category  without  necessarily  becoming  proficient  at  making  good  quality  choices  among  the 
alternatives.  For  example,  a  decision  maker  might  learn  product  information,  but  not 
develop  any  sense  of  attribute  importance  in  the  absence  of  a  need  to  evaluate  the 
alternatives.  Alternatively,  a  decision  maker  making  a  choice  in  one  category  may  develop 


^Anderson  (1983)   describes  a   similar  process. 


a  choice  rule  that  can  be  used  to  make  a  choice  in  another  category.  Thus,  decision  makers 
can  have  knowledge  of  alternatives  without  having  the  procedural  skill  of  a  choice  rule,  or 
vice  versa.  This  view  of  knowledge  is  represented  in  Figure  2.  Decision  makers  who  have 
both  alternative-specific  knowledge  and  rule  knowledge  would  be  classified  as  experts 
(Cell  4).  Decision  makers  with  neither  type  of  knowledge  would  be  classified  as  novices 
(Cell  1).  The  decision  makers  who  have  alternative-specific  knowledge,  but  not  procedural 
knowledge,  such  as  a  choice  rule,  would  be  found  in  Cell  2.  Decision  makers  with  rule 
knowledge,  but  not  alternative-specific  knowledge,  would  be  found  in  Cell  3. 


Figure  2  about  here. 


This  framework  extends  previous  research  by  directly  focusing  upon  the  stages  between 
being  a  novice  and  being  an  expert  We  propose  that  the  inclusion  of  two  intomediate 
categories  b^ween  those  of  novice  and  expert  more  realistically  depicts  the  range  of 
knowledge  that  may  be  possessed  by  decision  makers. 

In  the  studies  described  below,  we  examine  the  behaviors  of  decision  makers  in  the 
gray  area  between  being  a  novice  and  being  an  expert  Our  goal  is  not  to  establish  that 
decision  makers  obtain  expertise  by  passing  in  order  through  the  four  cells,  but  to 
understand  the  two  intermediate  stages  and  the  differences  between  them.  To  do  so,  we 
focus  on  the  systematic  differences  between  alternative-specific  knowledge  and  rule 
knowledge.  In  the  next  section,  we  present  hypotheses  about  the  nature  of  outcome 
performance,  or  choice  quality,  and  the  decision  maker's  perception  of  performance,  with 
respect  to  the  presence  and/or  absence  of  these  two  components  of  knowledge. 


HYPOTHESES 

There  are  two  key  aspects  of  knowledge  that  are  worthy  of  attention.  One  is  the 
possession  of  knowledge  by  decision  makers  and  the  other  is  the  recognition  that 
knowledge  is  possessed.  We  propose  that  in  route  to  becoming  experts,  decision  makers 
not  only  develop  a  knowledge  base,  but  that  they  also  become  aware  of  this  growing 
knowledge  base,  which  gives  them  more  confidence  in  their  decisions.  While  the  existence 
of  a  broader  knowledge  is  expected  to  manifest  itself  in  choices  of  superior  quality,  it  is  the 
awareness  of  the  knowledge  base  that  is  expected  to  influence  perceptions  of  the  quality  of 
the  choice. 

In  this  section,  hypotheses  about  the  influence  of  alternative-specific  knowledge 
and  of  rule  knowledge  are  presented.  The  effects  of  these  components,  both  singly  and 
together,  are  predicted  in  terms  of  objective  and  perceived  choice  quality.  Objective  choice 
quality  is  simply  how  good  the  chosen  alternative  is,  compared  with  other,  not  chosen, 
alternatives.  Perceived  choice  quality  refers  to  the  decision  maker's  perceptions  of  the 
choice  quality. 

Objective  Choice  Quality 

Psychology  research  in  expertise  suggests  that  experts  have  greater  access  to  strong 
methods  (i.e.,  procedures  or  rules)  for  solving  problems,  such  as  making  good  choices, 
than  do  novices,  who  rely  on  weak  methods  (Langley,  Simon,  Bradshaw  and  Zjrtgow 
1987;  Sweller,  Mawer  and  Ward  1983).  For  example,  in  physics  problem-solving,  these 
stronger  methods  make  use  of  axioms  or  fundamental  principles  to  solve  problems  (Chi,  et 
al.  1981;  Larkin,  et  al.  1980).  Novices  use  weaker  methods,  such  as  relying  on  superficial 
aspects  of  the  problem,  to  guide  solution  attempts.  These  weaker  methods  often  lead  to 
inferior  solutions.  For  decision  making,  the  idea  of  weak  and  strong  methods  suggests  that 
a  decision  maker  who  has  knowledge  of  an  appropriate  rule  (a  strong  method)  for 
integrating  and  evaluating  information  about  alternatives  will  tend  to  make  objectively  better 


choices  than  a  decision  maker  without  rule  knowledge.  In  the  classification  in  Figure  2, 
decision  makers  in  Cells  3  and  4  have  knowledge  of  choice  rules,  whereas  decision  makers 
in  Cells  1  and  2  do  not.  Therefore,  despite  knowledge  of  alternative-specific  information, 
Cell  2  decision  makers,  who  do  not  have  knowledge  of  choice  rules,  will  have  to  choose 
randomly  or  construct  an  appropriate  choice  rule,  unlike  decision  makers  in  Cells  3  and  4, 
who  simply  have  to  retrieve  an  appropriate  choice  rule.  Because  a  constructed  choice  rule 
does  not  provide  the  decision  maker  with  any  knowledge  of  how  effective  the  rule  might  be 
(i.e.,  there  is  no  past  history  of  success),  these  ad  hoc  rules  may  tend  overall  to  be  less 
effective  than  retrieved  rules.  Therefore,  we  expect  that  decision  makers  in  Cells  3  and  4 
will  objectively  outperform  decision  makers  in  Cells  1  and  2. 

It  should  be  noted,  however,  that  work  on  implicit  learning  (e.g.,  Gordon  and 
Holyoak  1983)  indicates  that  people  do  internalize  information  about  patterns  and  structure 
of  information,  and  that  they  may  use  this  information  to  abstract  rules  for  using 
information.  Therefore,  is  it  possible  that  Cell  2  decision  makers,  while  attending  to 
alternative  information,  may  unconsciously  abstract  choice  rules.  Thus,  decision  makers  in 
Cell  2  are  expected  to  have  an  advantage  over  decision  makers  in  Cell  1  in  their  ability  to 
organize  information.  Therefore,  we  also  expect  that  the  objective  performance  of  decision 
makers  in  Cell  2  should  be  better  than  that  of  decision  makers  in  Cell  1. 

Perceived  Chpifjtf^  Q'l'^h'ty 

Alternative-specific  knowledge  and  rule  knowledge  are  also  expected  to  influence 
perceptions  about  choice  processes  and  outcomes.  In  general,  we  expect  that  the  degree  of 
perceived  expertise  will  influence  decision  makers'  perceptions  of  choice  quality.  In  other 
words,  decision  makers  who  consider  themselves  to  have  a  broad  knowledge  base  will 
tend  to  rate  their  performance  more  highly  than  those  who  consider  themselves  to  possess  a 
more  limited  knowledge  base. 


Decision  makers  can  assess  their  degree  of  self-expertise  by  looking  at  either 
external  or  internal  indicators.  The  external  indicators  might  be  task-oriented,  such  as  the 
quality  of  the  choice  (i.e.,  outcome  feedback).  Internal  indicators  might  be  person- 
oriented,  such  as  the  recognition  that  one  possesses  procedural  skills  for  performing  a  task. 
In  general,  external  indicators  would  be  useful  where  the  correctness  or  incorrectness  of  the 
outcome  is  unambiguous,  as  in  situations  where  feedback  is  immediately  available.  In 
many  choice  situations,  however,  outcome  feedback  is  not  so  readily  available.  In  these 
situations,  decision  makers  may  rely  more  heavily  on  internal  indicators,  such  as  the 
recognition  that  they  possess  choice  skills,  or  on  process  feedback,  such  as  the  effort 
expended  in  making  the  choice  (Greyer,  Bettman  and  Payne  1989). 

There  is  some  evidence  that  the  internal  indicators  may  be  difficult  to  assess. 
Studies  have  shown  that  people  who  internalize  procedural  knowledge  through  practice  at  a 
task  may  not  be  aware  or  able  to  articulate  that  they  possess  such  knowledge  (Lewicki 
1986;  Lewicki,  Hill  and  Bizot  1988).  Rule  knowledge  may  be  encoded  at  a  deeper  level 
than  alternative-specific  knowledge,  and  thus  may  not  be  easily  recognized.  In  contrast, 
people  are  more  likely  to  be  aware  that  they  possess  alternative-specific  knowledge. 
Therefore,  cell  2  decision  makers,  who  have  alternative-specific  knowledge,  are  expected 
to  see  themselves  as  more  expert  than  cell  3  decision  makers. 

Note  that  the  interesting  difference  between  predictions  of  objective  and  subjective 
performance  lies  in  the  predictions  about  cells  2  and  3.  Cell  3  decision  makers  are  expected 
to  perform  objectively  better  than  cell  2  decision  makers,  but  cell  2  decision  makers  are 
expected  to  perceive  their  performance  more  highly  than  cell  3  decision  makers. 

To  summarize  the  expectations  about  performance,  we  believe  that  subjects  with 
rule  knowledge  will  outperform  subjects  without  rule  knowledge,  regardless  of  alternative- 
specific  knowledge.  In  terms  of  perceived  performance,  however,  we  expect  that  subjects 
with  altemative-^)ecific  knowledge  will  overestimate  their  performance,  placing  greater 
value  on  this  knowledge  than  is  warranted.  In  addition,  when  only  one  component  of 
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knowledge  is  present,  we  expect  that  rule  knowledge  will  prove  more  valuable  than 
alternative-specific  knowledge  in  terms  of  objective  choice  quality,  but  that  alternative- 
specific  knowledge  will  result  in  higher  estimates  of  perceived  performance  than  rule 
knowledge. 

Performance  Estimation  Error  Index 

In  addition  to  actual  and  perceived  performance,  a  performance  estimation  error 
index  can  be  developed  to  reflect  the  mismatch  betweai  objective  and  perceived  choice 
quality.  This  measure  is  constructed  by  subtracting  objective  choice  quality  ftom  perceived 
choice  quality  (Arkes,  Dawes  and  Christensen  1986).  This  index  is  a  measure  of 
overconfidence.  An  index  value  greater  than  zero  implies  overconfidence  in  performance, 
whereas  an  index  value  less  than  zero  implies  underconfidence.  Following  the  general 
arguments  outlined  earlier,  we  expect  decision  makers  in  cell  2  to  be  over  confident  and  in 
cell  3  to  be  underconfident  about  the  quality  of  their  choices.  We  also  expect  that  decision 
makers  in  cell  4,  who  presumably  have  the  greatest  awareness  of  the  knowledge  they 
possess,  to  be  closest  to  zero,  that  is,  least  likely  to  be  over-  or  underconfident 

METHOD 

The  hypotheses  were  tested  in  two  studies.  In  both  studies,  subjects  were 
undergraduates  at  a  major  midwestem  university.  All  subjects  received  extra  course  credit 
for  participating  in  the  studies.  Fifty-one  subjects  participated  in  the  first  study  and  ninety- 
seven  participated  in  the  second  study. 

Study  One 

Procedure  ami  Design,  Subjects  completed  a  pencil  and  paper  task  which  took 
approximately  thirty  minutes.  The  task  consisted  of  two  phases:  a  training  phase  and  a  test 
phase.  The  phases  are  described  below. 


The  test  phase  was  exactly  the  same  for  all  subjects.  Subjects  were  provided  with 
information  about  four  laptop  computers  on  four  attributes  in  a  brand/attribute  matrix. 
They  were  then  asked  to  choose  the  best  brand.  Laptop  computers  were  selected  after 
pretesting  indicated  that  subjects  had  very  little  knowledge  of  laptops.  Unfamiliarity  was 
necessary  in  order  to  establish  a  baseline  of  knowledge  about  the  alternatives  in  the 
category.  On  a  seven-point  scale,  the  mean  response  for  familiarity  with  the  category  was 
2.74  (where  1  was  Not  At  All  Familiar  and  7  was  Very  Familiar).  The  standard  deviation 
was  1.64,  and  the  median  response  was  2.00. 

Prior  to  the  test  phase,  subjects  went  through  a  training  phase.  The  purpose  of  the 
training  phase  was  to  achieve  the  manipulation  of  the  independent  variables.  Two 
independent  variables  were  manipulated  between-subjects:  alternative-specific  knowledge 
and  rule  knowledge.  Each  variable  was  manipulated  at  two  levels  (yes/no)  resulting  in  four 
treatment  conditions,  each  of  which  corresponded  to  a  cell  in  Figure  2.  Thus,  there  were 
four  different  training  conditions.  Each  subject  was  assigned  to  one  of  the  conditions. 
Subjects  in  condition  one  received  neither  alternative-specific  knowledge  nor  rule 
knowledge  (Cell  1).  Subjects  in  condition  two  received  rule  knowledge  but  not  alternative- 
specific  knowledge  (Cell  2).  Subjects  in  condition  three  received  alternative-specific 
knowledge  but  not  rule  knowledge  (Cell  3).  Subjects  in  condition  four  received  both 
alternative-specific  knowledge  and  rule  knowledge  (Cell  4). 

The  training  phase  consisted  of  five  problems.  In  each  problem,  information  on 
four  brands  was  provided  in  a  four-by-four  brand/attribute  matrix.  Based  on  the 
information,  subjects  were  given  a  task  to  complete  before  moving  on  to  the  next  problem. 

Figure  3  is  a  schematic  representation  of  the  training  and  the  test  phase  for  the  four 
cells.  Subjects  assigned  to  Cell  1  were  given  information  on  four  brands  of  microwave 
ovens  and  asked  to  judge  the  similarity  of  the  brands  in  the  training  phase.  It  was  expected 
that  they  would  abstract  a  rule  to  judge  similarities  of  brands  by  the  end  of  five  trials.  It 
was  also  expected  that  they  would  obtain  alternative-specific  knowledge  about  microwaves. 
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In  the  test  phase,  then,  these  subjects  would  have  neither  rule  knowledge  for  making  a 
choice  nor  alternative-specific  knowledge  for  laptop  computers  (because  they  were  given 
microwave  information). 


Figure  3  about  here. 


Subjects  assigned  to  Cell  2  were  given  information  about  four  brands  of 
microwaves  and  asked  to  choose  the  best  brand  in  the  training  phase.  This  was  expected  to 
provide  them  with  rule  knowledge  about  making  the  best  choice,  but  not  with  alternative- 
specific  knowledge  about  laptops.  Subjects  assigned  to  Cell  3  were  given  information 
about  laptops  and  asked  to  judge  similarity  between  brands.  Finally,  subjects  assigned  to 
Cell  4  were  given  laptop  computers  and  asked  to  choose  the  best  brand  in  the  five  training 
trials. 

The  goal  of  the  training  phase  was  to  provide  subjects  with  an  opportunity  to 
abstract  a  rule.  Anzai  and  Simon  ( 1979)  suggest  that  methods  may  be  developed  during 
multiple  attempts  to  reach  a  goal.  Therefore,  subjects  who  have  multiple  opportunities  to 
make  a  choice  are  more  likely  to  abstract  an  appropriate  choice  rule  than  subjects  who  make 
a  choice  only  once,  in  the  test  phase.  Also,  subjects  who  had  multiple  exposures  to  the 
product  category  were  expected  to  develop  familiarity  with  the  relevant  attributes  and  their 
values  (Coupey  and  Nakamoto  1988). 

Regardless  of  the  condition,  all  problems  were  constructed  to  be  comparable  in  the 
number  of  brands  and  attributes  subjects  saw.  Each  problem  had  four  brands  and  four 
attributes.  For  the  microwave  problems,  the  attributes  were  interior  capacity,  number  of 
power  levels,  wattage,  and  length  of  warranty.  For  laptop  computers,  the  attributes  were 
weight,  number  of  fHX)grams,  internal  memory,  and  external  memory.  All  attributes  were 
selected  from  the  sets  of  features  used  by  Consumer  Reports  to  rate  microwaves  and 
laptops. 
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Dependent  measures.  Two  primary  dependent  measures  were  obtained:  an  objective 
measure  of  choice  quality  and  subjects'  perceived  measure  of  choice  quality.  The  objective 
measure  was  the  rank  order  data  of  the  subject's  selected  brand  from  the  final  four  brands 
in  the  test  problem  (see  Stone  and  Schkade  (1992)  for  a  similar  measure  of  performance). 
The  brands  were  ranked  from  the  best  to  the  worst  using  a  weighted  adding  rule.  This  rule 
has  been  used  in  previous  research  (cf.  Johnson  and  Payne  1985)  to  establish  an  upper  end 
of  choice  performance  The  ranks  were  computed  using  the  attribute  weights  obtained  from 
each  subject  at  the  end  of  the  test  phase.  These  weights  were  multiplied  with  attribute 
levels,  for  all  attributes  of  a  brand.  The  resulting  products  were  summed  across  each  brand 
to  obtain  the  total  brand  value.  The  brand  with  the  highest  value  was  ranked  1  (indicating 
best  performance),  and  the  brand  with  the  lowest  value  was  ranked  4  (indicating  worst 
performance). 

The  perceived  choice  quality  was  simply  the  subject's  own  estimate  of  the  rank  for 
his  or  her  chosen  brand.  Subjects  were  asked  to  check  the  statement  that  they  felt  reflected 
their  choice  quality:  "I  chose  the  best  brand,"  "I  chose  the  second  best  brand,"  etcetera.  A 
lower  value  indicated  a  superior  assessment  of  performance. 
Results, 

Objective  Performance.  A  two-way  analysis  of  variance  with  rule  knowledge  and 
alternative-specific  knowledge  as  factors  revealed  a  significant  main  effect  of  rule 
knowledge  (F(l,50)  =  6.64;  p<0.01).  Subjects  who  made  choice  decisions  in  the  training 
phase  ouQ)erformed  subjects  who  made  similarity  judgments  in  the  training  phase  (mean 
ranks  of  1.7  and  2.36,  respectively).  There  was  no  significant  effect  of  alternative-specific 
knowledge  (F(140)  =  0.04;  p<0.83),  or  the  interaction  effect  of  rule  and  alternative- 
specific  knowledge  (F(l,50)  =  0.03;  p<0.85).  However,  because  there  was  an  a  priori 
theoretical  basis  for  expecting  that  the  cell  means  would  differ  in  a  predicted  pattern,  a 
statistical  contrast  of  the  means  for  cells  2  and  3  was  performed  (Winer,  1971,  pg.  384). 
The  contrast  approached  significance  (t  =  1.66;  p<0.10),  suggesting  that  knowledge  of  a 
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choice  rule  may  be  more  helpful  in  making  good  quality  choices  than  knowledge  about 
alternatives.  This  finding  provides  tentative  support  for  the  premise  that  it  is  essentially 
rule  knowledge  that  results  in  superior  performance. 

Perceived  Choice  Quality.  The  results  for  perceived  choice  quality  show  that  the 
pattern  is  the  reverse  of  that  found  for  objective  choice  quality;  highest  estimates  of 
performance  were  given  by  subjects  in  cell  1  (mean  =  1.25).  Cell  2  subjects  were  next 
(mean  =  1.56),  followed  by  cell  3  subjects  (mean  =  1.60).  Subjects  in  cell  4  (the  experts) 
gave  the  lowest  estimates  (mean  =  2.00). 

A  two-way  ANOVA  with  rule  knowledge  and  alternative-specific  knowledge  as 
factors  revealed  a  significant  main  effect  of  rule  knowledge  (F(l,50)  =  4.86;  p  <  0.03); 
subjects  with  rule  knowledge  (mean  =  1.83)  perceived  their  performance  more  negatively 
than  those  with  no  rule  knowledge  (mean  =  1.43).  There  was  also  a  significant  main  effect 
of  alternative-specific  knowledge  (F(l,50)  =  4.0;  p  <  0.05);  subjects  with  alternative- 
specific  knowledge  assessed  their  performance  more  negatively  (mean  =  1.76)  than 
subjects  without  alternative-specific  knowledge  (mean  =1.41).  The  interaction  effect  was 
not  significant  (F(l,50)  =  0.06;  p  <  0.81).  The  contrast  between  cells  2  and  3  was  also 
insignificant  (t  =  .88;  p  <  0.88). 

Performance  Estimation  Error.   An  index  of  performance  estimation  CTror  was 
constructed  by  subtracting  perceived  choice  quality  estimates  ftom  objective  performance 
ranks^.  In  this  index  values  greater  than  zero  indicate  overconfidence,  and  values  less  than 
zero  indicate  underconfidence.  From  the  index  it  appears  that  cell  1  subjects  were  most 
overconfident  (mean  =  1.17),  while  cell  4  subjects  were  actually  underconfident  (mean  =  - 
.31).  As  hypothesized,  cell  2  subjects  (mean  =  0.75)  were  more  overconfident  than  cell  3 


^Note  that  this  measure  is  actually  the  negative  of  the  overconfidence  measure  defined  by 
Arkes  et  al.  (1986).    This  was  done  to  account  for  the  fact  that  the  performance  indices 
in  this  study  were  ranks,  where  a  higher  value  indicated  a  lower  level  of  performance. 
By  taking  the  negative  of  tiie  Arkes  et  al.  definition,  we  ensured  that  a  higher 
performance  estimation  error  reflected  more  overconfidence. 
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subjects  (mean  =  0.1).  Cell  3  subjects  were  closest  to  predicting  their  performance  levels 
accurately. 

An  analysis  of  variance  with  rule  and  alternative-specific  knowledge  as  the 
between- subjects  fectors  revealed  a  significant  main  effect  of  rule  knowledge  (F(i40)  = 
13.06,  p  <  .0007).  The  mean  performance  estimation  error  index  for  subjects  without  rule 
knowledge  was  0.93,  compared  with  -0.13  for  subjects  with  rule  knowledge.  There  was 
no  significant  effect  of  alternative- specific  knowledge  (F(i^o)  =  1-97,  p  <  .16),  or  of  the 
interaction  (F(l,50)  =  0.0002;  p  <  0.99).  The  contrast  between  cells  2  and  3  was  also 
insignificant  (t  =  1.56;  p  <  0.13). 

The  objective  and  perceived  choice  quality,  as  well  as  the  performance  estimation 
errors  for  Study  1  are  summarized  by  cells  in  Figure  4. 


Figure  4  about  here. 


Discussion  Study  1  enabled  an  examination  of  the  benefits  of  alternative-specific 
knowledge  relative  to  rule  knowledge.  The  results  of  Study  1  indicate  that  being  able  to 
use  a  known  choice  rule,  or  procedural  knowledge,  appears  to  be  more  useful  in  making 
good  choices  than  does  alternative-specific  knowledge.  Subjects  with  alternative-specific 
knowledge  have  a  small  advantage  over  subjects  who  have  neither  rule  nor  alternative- 
specific  knowledge.  Moreover,  having  rule  knowledge,  even  in  the  absence  of  alternative- 
specific  knowledge,  is  still  more  beneficial  than  just  alternative-specific  knowledge. 

^.ifni^ti^n*^,  Two  different  rationales,  one  theoretical  and  one  procedural,  may 
explain  this  finding.  First,  subjects  with  rule  knowledge  may  have  used  information  more 
consistentiy  than  subjects  without  rule  knowledge.  This  may  have  led  to  more 
compensatory  processing  of  attribute  values,  thereby  resulting  in  better  objective  choice 
quality.  Subjects  without  rule  knowledge  may  not  have  been  able  to  use,  or  even  to 
construct,  a  compensatory  rule  very  well.  In  essence,  this  rationale  assumes  that  subjects 
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with  only  alternative- specific  knowledge  either  chose  randomly,  or  that  they  attempted  to 
construct  rules  on-the-spot  If  the  latter  assumption  is  true,  then  the  constructed  rules  were 
not  as  optimal  for  making  choices  as  the  rules  abstracted  by  subjects  from  multiple  choice 
episodes  during  the  training  phase. 

The  second  possible  rationale  for  why  ceU  3  subjects  outperformed  ceU  2  subjects 
may  lie  in  the  experiment  procedure.  All  of  the  stimuli  were  presented  as 
alternative/attribute  matrices.  This  fonnat  may  have  facilitated  the  use  of  a  compensatory 
rule  that  promoted  better  choices  more  than  it  helped  subjects  leam  and  organize  product 
information.  One  benefit  of  knowledge  in  developing  expertise  is  that  it  may  help  the 
decision  maker  to  structure  information  (Beattie  1983),  in  effect,  to  construct  a  useful 
representation  of  the  choice.  One  limitation  of  Study  1  was  that  the  manipulation  of 
alternative-specific  knowledge  did  not  enable  examination  of  this  representational  benefit  of 
knowledge. 

A  second  fector  which  limits  the  conclusions  which  may  be  drawn  from  this  study 
is  the  lack  of  control  over  rule  abstraction.  For  example,  subjects  in  cells  1  and  2  might 
have  developed  choice  rules  on  the  final  decision.  There  was  no  time  limit  imposed,  so 
subjects  may  have  tried  out  different  methods  for  evaluating  brands,  perhaps  using  process 
feedback,  such  as  effort  (Greyer  et  al.  1989),  as  a  guide  until  an  acceptable  rule  was 
developed  Thus,  having  unlimited  time  to  try  out  rules  may  have  enabled  subjects  in  cells 
1  and  2  to  acquire  some  of  the  procedural  skills  presumed  to  be  available  only  to  subjects  in 
cells  3  and  4.  In  gaieral,  the  differences  between  objective  performance  in  the  four  cells 
were  smalL  Although  the  cell  means  follow  the  predicted  pattern  for  objective 
performance,  we  cannot  unequivocally  state  that  the  observed  differences  are  systematic 
and  would  not  change  with  changes  to  the  manipulations  or  the  inclusion  of  additional 
controls. 

Finally,  while  this  study  was  designed  to  facilitate  comparison  of  rule  knowledge 
and  alternative- specific  knowledge  in  cells  2  and  3,  it  could  not  be  used  to  examine  the 
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effects  that  awareness  of  rule  knowledge  might  have  on  perceived  performance.  Subjects 
with  rule  knowledge  may  not  have  recognized  that  they  had  a  useful  rule,  obtained  through 
repeated  exposures.  Work  on  implicit  learning  (e.g.,  Reber  1976)  suggests  that  knowledge 
—  as  of  a  rule  ~  and  the  ability  to  use  it,  may  often  precede  awareness  of  the  knowledge 
and  the  ability  to  verbalize  it.  Thus,  it  is  expected  that  subjects  who  are  aware  that  they 
have  an  appropriate  rule  for  making  a  choice  will  be  more  confident  in  their  choices  than 
subjects  who  are  not  made  aware  that  they  have  an  appropriate  rule  for  choice. 

A  second  study  was  designed  to  examine  the  role  of  awareness  on  perceived  choice 
quality,  and  to  address  the  limitations  discussed  earlier.  To  this  end,  changes  were  made  to 
the  manipulations  of  alternative-specific  and  rule  knowledge,  the  format  of  the  stimuli,  and 
the  procedure  used  to  collect  the  data. 

Study  Two 
Procedure  and  Design 

Subjects  completed  a  paper  and  pencil  task  which  took  approximately  thirty 
minutes.  As  in  Study  One,  subjects  made  a  brand  choice  and  ranked  their  perceived 
performance. 

The  study  was  a  2x2  between-subjects  design.  The  independent  variables  of 
alternative-specific  information  and  rule  knowledge  were  manipulated,  resulting  again  in 
four  treatment  conditions.  However,  in  order  to  address  the  limitations  of  the  first  study 
described  above,  and  to  assess  the  impact  of  awareness  of  a  knowledge  component  on 
choice  behavior,  both  manipulations  were  altered  as  described  below. 

The  alternative-specific  knowledge  manipulation  was  changed  primarily  in  two 
ways.  First,  subjects  did  not  develop  alternative-specific  knowledge  repeated  exposure. 
Second,  the  alternative/attribute  training  format  was  discarded  Instead,  to  provide  a  more 
stringent  and  realistic  assessment  of  the  benefits  of  alternative-specific  knowledge,  subjects 
in  the  alternative-specific  knowledge  condition  received  a  one  page  description  of  the 
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product  category,  laptop  computers.  Without  referring  to  specific  brands,  the  page  detailed 
which  attributes  were  relevant  for  making  good  choices,  and  the  usual  ranges  and  most 
typical  values  of  the  attributes.  Six  attributes  were  used:  memory,  battery  life,  weight,  disk 
speed,  screen  quality,  and  whether  the  laptop  had  a  monitor  port.  Information  about  which 
attributes  were  not  diagnostic  was  also  provided.  For  example,  statements  such  as, 
'*Weight  of  the  laptop  computer  is  also  important,  but  because  all  laptops  tend  to  weigh 
pretty  much  the  same  amount,  this  information  is  not  helpful  in  making  a  choice,"  were 
included.  Subjects  in  the  no  alternative-specific  knowledge  condition  were  given  a  page  of 
information  of  the  same  length  and  complexity,  but  for  microwave  ovens.  In  both 
conditions,  the  test  phase  required  subjects  to  make  a  choice  among  laptop  computers. 

The  second  change  to  the  alternative  knowledge  manipulation  was  in  the 
presentation  of  the  test  stimulus.  Rather  than  presenting  the  information  in  matrix  form,  the 
information  was  presented  in  paragraph  form,  one  paragraph  per  brand.  This  presentation 
better  reflects  the  way  information  about  products  is  encountered  in  many  purchase 
decisions,  both  in  terms  of  the  sequential,  brand-by-brand  nature  of  information 
availability,  and  in  terms  of  the  structure  of  the  information  about  attributes  and  their 
values. 

In  order  to  assess  the  impact  of  being  aware  of  a  choice  rule  on  choice  quality  and 
perceptions  of  the  choice,  the  rule  knowledge  manipulation  was  also  changed  One-half  of 
the  subjects  (rule  knowledge  condition)  were  trained  in  the  use  of  a  choice  rule  that  they 
were  told  would  result  in  the  selection  of  the  best  overall  brand,  if  used  correctly.  The 
remaining  subjects  (no  rule  condition)  were  given  no  training. 

Subjects  in  the  rule  knowledge  condition  were  taught  to  use  a  simple  choice  rule. 
The  rule  consisted  of  the  following  steps.  First,  the  subjects  had  to  rank  order  the  brands 
on  each  attribute  from  1  (the  best)  to  4  (the  worst).  Second,  subjects  were  asked  to  sum 
the  ranks  across  the  brands.  The  brand  with  the  lowest  total  score  was  the  best  brand. 
This  strategy  was  selected  because  of  its  intuitive  appeal  and  simplicity;  moreover,  because 
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the  rule  is  compensatory,  it  is  a  reasonably  normative  method  for  making  a  choice.  The  test 
stimulus  was  constructed  so  that  one  brand  was  always  the  clear  winner  under  this 
strategy.  In  addition,  the  stimulus  was  constructed  so  that  the  brand  rankings  obtained 
with  this  strategy  would  be  the  same  as  those  obtained  using  a  more  complex  weighted- 
adding  strategy.  Note  that  this  manipulation  should  result  in  an  objective  performance 
advantage  for  subjects  in  the  rule  knowledge  condition,  thus  limiting  conclusions  that  may 
be  drawn  about  the  performance  of  subject  with  rule  knowledge  relative  to  the  performance 
of  subjects  with  alternative-specific  knowledge.  In  contrast  to  Study  1,  the  goal  of  Study  2 
is  to  examine  the  effect  of  knowledge  awareness  on  overconfidence,  the  influence  of  the 
two  knowledge  types.  Thus,  attention  here  is  focused  on  subjects'  perceptions  of 
performance  when  a  ceiling  for  objective  performance  is  induced  by  rule  training  and  when 
subjects  are  made  aware  that  they  have  an  appropriate  rule. 

A  time  limit  was  also  imposed  on  all  subjects.  The  time  limit  was  introduced  to 
reduce  the  possibility  that  subjects  in  the  no  rule  condition  had  sufficient  time  to  construct  a 
rule  during  the  test  phase.  The  time  limit  was  determined  in  a  pretest  by  obtaining  an 
average  of  the  response  times  for  subjects  adept  in  the  use  of  the  summed  ranks  rule.  The 
average,  two  and  one-half  minutes,  was  used  as  a  cut-off  time  in  all  conditions.  Subjects 
were  not  told  that  there  was  any  time  constraint  before  they  began  the  test  phase.  When 
time  ran  out,  subjects  were  asked  to  make  a  choice  immediately,  and  the  experiment  was 
ended. 
Results. 

Objective  Performance:  Manipulation  Oieck.  The  results  of  a  two-way  analysis  of 
variance  served  as  manipulation  check  on  the  rule  knowledge  manipulation.  Th^e  was  a 
significant  effect  of  rule  knowledge  (F(l,96)  =  34.46;  p  <  0.0001).  Subjects  who  were 
taught  the  rule  significantiy  outperformed  those  who  were  not  taught  the  rule  (mean  choice 
ranks  were  U3  and  2.86,  respectively).  Therefore,  the  rule  manipulation  was  successful. 
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A  contrast  between  cells  2  and  3  on  the  dependent  measure  of  objective  choice 
quality  was  also  significant  (t  =  3.72,  p  <  0.003),  thereby  replicating  the  finding  from 
Study  1.  The  overall  performance  pattern  largely  mirrored  that  observed  in  Study  1; 
subjects  in  cell  1  performed  worst  (mean  =  2.96),  followed  by  subjects  in  cells  2  (mean  = 
2.76),  3  (mean  =  1.56),  and  4  (mean  =  1.5). 

Perceived  Choice  Quality.  The  results  of  a  two-way  ANOV A  with  perceived 
performance  as  the  dependent  measure  were  used  to  examine  expectations  about  the  effect 
of  awareness  of  rule  knowledge  on  performance.  Rule  knowledge  did  have  a  significant 
effect  on  subjects'  perceptions  of  their  performance  (F(l,  96)  =  3.99;  p  <  0.04).  As 
predicted,  the  pattern  of  perceived  performance  reversed  from  Study  1  to  Study  2.  In 
Study  2,  subjects  who  were  trained  in  the  use  of  a  rule,  and  who  were  aware  of  its 
benefits,  not  only  made  better  choices  objectively,  but  also  had  confidence  in  the  quality  of 
their  choices.  Subjects  with  rule  knowledge  and  awareness  of  such  knowledge  had  a  mean 
perceived  performance  of  1.32,  compared  with  1.50  for  subjects  without  rule  knowledge. 
Recall  diat  for  Study  1,  the  comparable  values  were  1.83  and  1.43. 

There  was  no  significant  effect  of  alternative-specific  knowledge  (F(l,  96)  =  0.08; 
p  <  0.78),  but  the  interaction  effect  approached  significance  (F(l,  96)  =  3.14;  p  <  0.08). 
The  pattern  of  means  was  roughly  the  reverse  of  that  observed  in  Study  1.  Subjects  in  cell 
1  ranked  their  performance  lowest  (mean  =  1.6),  followed  by  subjects  in  cell  2  (mean  = 
1.4).  Contrary  to  expectations,  however,  subjects  in  cell  3  ranked  their  performance  more 
highly  than  subjects  in  cell  4  (means  of  1.26  and  1.38,  respectively).  A  contrast  of  the 
difference  between  subjects  in  cells  2  and  3  was  not  significant  (t  =  0.88;  p  <  0.38). 

Performance  Estimation  Error.  The  results  were  fairly  consistent  with  those 
obtained  in  Study  1,  and  again  provided  support  for  predictions  about  overconfidence. 
Subjects  in  cells  1  and  2,  with  means  of  1.36  each,  were  most  overconfident  about  their 
performance.  As  expected,  subjects  in  cell  3  (mean  =  .30)  were  less  confident  than 
subjects  in  cell  2.  Cell  4  subjects  were  the  least  overconfident  (mean  =  0.12).  Despite  an 
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insignificant  interaction  of  rule  knowledge  and  alternative-specific  knowledge  (F(  1, 96)  = 
0.52;  p  <  0.47),  the  contrast  of  the  means  for  cells  2  and  3  was  significant  (t  =  2.96;  p  < 
0.004). 

A  two-way  ANOVA  also  revealed  a  significant  effect  of  rule  knowledge  (F(l,  96)  = 
20.38;  p  <  0.0001).  The  mean  for  subjects  without  rule  knowledge  was  1.36,  compared 
with  0.21  for  subjects  with  rule  knowledge.  Subjects  without  rule  knowledge  were 
overconfident,  whereas  subjects  with  rule  knowledge  were  less  overconfident  Alternative- 
specific  knowledge  did  not  have  a  significant  effect  on  overconfidence  (F(l,  96)  =  0.12;  p 
<  0.72). 

These  results  of  Study  2  are  summarized  in  Figure  5. 


Figure  5  about  here. 


Discussion.  Study  2  examined  the  effects  of  knowledge  awareness  on  choice  performance. 
It  also  addressed  several  design  limitations  of  Study  1 ;  subjects  were  presented  with  more 
realistic  stimuli,  and  a  time-limit  was  imposed  to  more  effectively  manipulate  rule 
knowledge. 

The  results  of  Study  2  are  similar  to  those  of  Study  1 ,  with  the  exception  of  the 
results  for  perceived  performance.  The  similarity  in  result  patterns  across  cells,  given  the 
differences  in  independent  variable  manipulations,  lends  stroigth  to  the  assertion  that  there 
are  two  independent  knowledge  components  operating  in  the  evaluation  stage  of  the  choice 
process:  rule  knowledge  and  alternative-specific  knowledge.  The  relative  influence  of  these 
knowledge  types  on  perceived  performance  appears  to  differ  primarily  as  a  function  of  rule 
awareness. 

Making  subjects  aware  of  an  appropriate  rule  for  making  a  choice  has  an  effect  on 
perceived  performance.  In  Study  1,  subjects  without  an  opportunity  to  develop  a  rule  over 
repeated  exposures  to  the  product  category  rated  their  performance  more  highly  than  those 
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subjects  with  such  an  opportunity.  In  Study  2,  the  pattern  reverses;  subjects  trained  in  the 
use  of  a  rule  rated  their  performance  more  highly  than  those  not  trained  in  rule  use. 

The  Performance  Estimation  Error  index  suggests  that  one  effect  of  rule  awareness 
is  to  make  subjects  bettCT  able  to  judge  their  choice  performance.  Subjects  without  the  rule 
training  tended  to  be  significantly  less  able  to  gauge  their  performance,  and  they  tended  to 
be  more  overconfident  than  subjects  with  the  rule  training. 

GENERAL  DISCUSSION 

These  two  studies  reveal  a  broader  view  of  the  knowledge  components  influential  in 
choosing  alternatives  than  has  been  previously  addressed.  In  this  paper,  knowledge  in  the 
evaluation  stage  of  the  choice  process  has  been  conceptualized  as  a  multi-stage  process  in 
which  decision  makers  can  have  varying  levels  of  knowledge.  This  contrasts  with 
previous  approaches  in  which  decision  makers  have  tended  to  be  categorized  as  either 
novices  or  experts.  Independent  manipulation  of  the  two  knowledge  components,  rule 
knowledge  and  alternative-specific  knowledge,  provides  insights  into  the  relative  benefits 
of  each  knowledge  type.  A  focus  on  the  objective  and  perceived  performance  as  a  function 
of  combinations  of  these  two  knowledge  types  enables  conclusions  about  possible 
remedies  for  overconfidence  in  choice  behavior. 

To  summarize,  alternative-specific  knowledge  is  defined  as  knowledge  specific  to 
the  category  in  which  a  choice  is  being  made.  This  includes  knowledge  about  the  attributes 
relevant  for  making  choices,  and  the  usual  ranges  and  most  typical  values  of  these 
attributes.  This  type  of  task-specific  knowledge  is  relevant  only  to  choices  being  made 
fit>m  a  particular  category. 

Rule  knowledge  is  defined  as  task-general  knowledge.  This  type  of  knowledge 
may  consist  of  general  choice  rules  that  a  decision  maker  can  apply  to  a  variety  of  choice 
situations.  An  example  of  rule  knowledge  is  knowing  how  to  use  the  weighted  adding  rule 
to  evaluate  altoTiatives.  Rule  knowledge  can  be  applied  to  choice  in  any  category. 

21 


These  two  knowledge  components  were  used  to  conceptualize  the  various  stages  of 
knowledge  for  making  a  choice.  By  treating  each  knowledge  type  as  a  dichotomous 
variable,  four  unique  stages  of  knowledge  were  described.  Two  studies  were  conducted  to 
assess  the  validity  of  this  conceptualization,  by  examining  subjects'  performance  on  both 
objective  and  perceived  choice  quality. 

The  findings  were  generally  consistent  with  the  hypotheses.  Rule  knowledge 
tended  to  result  in  superior  objective  pCTformance,  whereas  altCTnative-specific  knowledge 
led  to  superior  perceived  performance.  Interestingly,  subjects  with  neither  rule  nor 
alternative-specific  knowledge,  and  who  peformed  the  worst  objectively,  also  tended  to 
perceive  their  performance  as  being  the  best  It  appears  that  one  aspect  of  knowledge  is  the 
development  of  a  more  realistic  assessment  of  one's  performance.  This  was  demonstrated 
by  the  performance  estimation  error  index. 

The  analysis  of  the  performance  estimation  error  index,  obtained  by  subtracting 
perceived  performance  from  objective  performance,  showed  that,  in  both  studies.  Cell  2 
subjects  (alternative-specific  knowledge,  no  rule  knowledge)  tended  to  be  mere 
overconfident  than  Cell  3  subjects  (rule  knowledge,  no  alternative-specific  knowledge).  A 
possible  explanation  for  this  finding  may  lie  in  the  accessibility  of  the  knowledge 
component  Altoiiative-specific  knowledge  may  be  more  accessible  than  rule  knowledge, 
which  may  be  more  deeply  encoded.  Some  support  for  this  view  was  found  in  Study 
Two.  When  subjects  were  made  aware  that  they  possessed  rule  knowledge,  their  perceived 
choice  quality  increased. 

T  imitatinns 

Several  limitations  to  this  research  must  be  noted  in  ordCT  to  evaluate  the 
contributions.  First,  both  experiments  were  conducted  in  artificial,  laboratory  settings. 
Rather  than  measuring  subjects'  knowledge  of  the  stimulus  category,  we  opted  to  use  a 
category  with  which  most  subjects  were  unfamiliar,  and  then  observe  the  growth  of 
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knowledge  and  its  effects  on  performance.  The  disadvantage  to  this  approach  is  that 
acquisition  of  the  knowledge  types  of  interest  may  not  perfectly  reflect  the  way  knowledge 
is  accumulated  over  time  and  used  in  everyday  decisions.  This  shortcoming  may  be 
particularly  true  in  subjects'  use  of  attribute  weights.  Even  with  multiple  exposures, 
subjects  may  not  have  internalized  and  used  attribute  weight  information  in  the  same  way  as 
they  would  use  preferences  for  attributes  developed  over  time  and  experience.  The 
alternative,  however,  had  disadvantages  that  we  deemed  to  outweigh  any  advantages.  For 
example,  it  is  not  clear  how  to  accurately  estimate  the  amount  and  structure  of  prior 
knowledge  a  subject  might  bring  to  a  choice  task.  Some  subjects  may  store  information  in 
previously  made  overall  evaluations,  while  others  might  store  attribute-level  information. 
These  different  storage  techniques  may  be  confounded  with  rule  knowledge  acquired  in 
previous  exposures  to  the  category.  Thus,  a  trade-off  was  made  of  realism  for  control. 

The  same  trade-off  was  made  in  the  manipulation  of  rule  knowledge.  Rule 
knowledge  was  achieved  in  Study  1  by  exposing  subjects  to  five  choice  situations  and  in 
Study  2  by  teaching  subjects  a  choice  rule.  In  reality,  decision  makers  probably  abstract 
rules  by  a  combination  of  the  two  methods,  i.e.,  by  making  repetitive  choices  and  learning 
choice  rules  from  others.  Moreover,  decision  makers  in  real  life  probably  have  a  repertoire 
of  choice  rules  with  a  mega-strategy,  sensitive  to  situational  constraints  (e.g.,  time 
pressure),  that  tells  them  when  to  use  which  choice  rule. 

The  second  limitation  to  be  noted  is  the  assumption  that  optimal  choice  is  predicted 
by  a  multiattribute,  weighted  adding  rule.  Although  such  rules  are  commonly  used  in 
decision  research,  they  do  have  constraints.  First,  they  cannot  reflect  cutoffs  a  decision 
maker  might  have  for  minimally  acceptable  levels  of  an  attribute.  Second,  these  types  of 
rules  are  misleading  if  the  attributes  are  correlated  (e.g.,  miles  per  gallon  and  size  in 
automobiles).  Because  we  used  a  category  with  which  subjects  tended  to  have  no 
familiarity,  however,  these  issues  were  less  problematic. 
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Despite  these  limitations,  the  studies  described  in  this  paper  provide  several  insights 
into  types  of  knowledge  and  their  influences  on  objective  and  perceived  performance.  In 
the  following  section,  a  discussion  of  the  conclusions  and  their  implications  is  provided. 

Conclusions 

Three  main  conclusions  may  be  drawn  from  the  results  of  this  research.  These  are: 
1)  that  knowledge  can  be  described  as  a  multi-stage  process,  2)  that  knowledge  may 
influence  objective  choice  quality,  but  awareness  of  knowledge  influences  perceived  choice 
quality,  and  3)  that  constructive  processes  are  ubiquitous  in  the  development  of  knowledge 
for  making  choices. 

Knowledge  as  a  Multi-type  Process.  In  this  paper,  knowledge  has  been  described  as 
consisting  of  several  types,  from  being  a  novice  to  being  an  expert,  although  evidence  is 
only  provided  for  four  discrete  types.  Decision  makers  may  not  pass  through  these  types 
in  order.  As  we  demonstrate  in  Study  2,  simultaneous  presentation  of  rule  and  alternative- 
specific  knowledge  can  create  instant  experts.  Mere  knowledge  that  these  different  types 
may  exist,  however,  can  be  used  by  both  decision  makers  and  those  who  present 
information  to  decision  makers.  By  recognizing  the  characteristics  which  identify  one  as  a 
member  of  a  particular  knowledge  category,  and  by  knowing  the  advantages  and  limitations 
of  each  category,  information  presentations  can  be  tailored  for  more  effective  choice 
behavior. 

Knowledge  and  Awareness.  The  results  of  Studies  1  and  2  indicate  that  while  knowledge 
influences  the  quality  of  choice,  it  is  the  awareness  of  knowledge  that  affects  the  decision 
maker*s  confidence  in  the  choice.  This  has  important  ramifications  for  public  policy 
makers.  For  example,  consider  the  finding  that  objective  and  perceived  perfOTmance  are 
inversely  related  unless  decision  makers  are  aware  of  their  knowledge.  Building  on  the 
conclusion  above,  the  potential  for  unsatisfactory  choices  can  be  reduced  by  making 
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decision  makers  in  Cells  1  and  2  aware  that  they  are  likely  to  overestiniate  the  quality  of 
their  choices. 

Consrructive  Processes.  Recently,  research  in  several  areas  has  focused  on  the 
constructive  development  of  preferences  and  the  strategies  used  to  determine  these 
preferences  (Payne,  Bettman,  Coupey,  and  Johnson  1992).  The  results  of  Study  1 
underscore  the  ubiquity  of  constructive  processes,  as  they  suggest  that  decision  makers 
without  the  opportunity  to  retrieve  or  develop  choice  rule  during  a  training  phase  may 
construct  choice  rules  in  a  single  test  case,  when  given  ample  time.  Because  no  outcome 
feedback  was  provided,  the  decision  maker's  decision  to  stick  with  a  constructed  rule  must 
be  driven  be  internal  indicators  of  performance  (i.e.,  process  feedback,  such  as  effort). 
That  subjects  in  the  no  rule  condition  perceived  their  choices  to  be  higher  in  quahty  than 
subjects  in  the  rule  condition  suggests  that  the  higher  amounts  of  effort  presumably 
required  of  no  rule  subjects  in  the  test  case  may  be  used  as  proxies  for  accuracy.  Paese  and 
Sniezek  ( 199 1 )  describe  a  similar  finding  of  effort  as  a  proxy  for  accuracy  in  a  judgment 
task-  Further  research  should  consider  the  effect  of  rule  awareness  as  a  mediator  of  these 
effort/accuracy  proxies. 
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Figure  2 
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Figure  5 
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